test(presets): silence expected UserWarnings in self-test composition…#2373
test(presets): silence expected UserWarnings in self-test composition…#2373Quratulain-bilal wants to merge 5 commits intogithub:mainfrom
Conversation
… tests
The self-test preset that ships with the repo provides a wrap-strategy
command (speckit.wrap-test) intentionally without a corresponding core
base layer, exercising the 'no base layer' branch of
_reconcile_composed_commands().
Eighteen tests across TestSelfTestPreset and TestPresetSkills install
this preset and trigger an expected UserWarning. Running the suite with
-W error::UserWarning surfaces them as test noise that could obscure
unrelated warnings.
Add class-level pytest.mark.filterwarnings filters to acknowledge the
two known messages ('Cannot compose command speckit.wrap-test' and
'Post-install reconciliation failed for self-test') so other UserWarning
sources still propagate normally.
Fixes github#2363
There was a problem hiding this comment.
Pull request overview
Reduces test-suite noise in preset composition tests by explicitly acknowledging two expected UserWarnings emitted when installing the repo’s self-test preset (which intentionally includes a wrap-strategy command without a base layer).
Changes:
- Add class-level
pytest.mark.filterwarnings(...)entries to silence two known, expected warning messages. - Expand docstrings on the affected test classes to document why the warnings are expected.
Show a summary per file
| File | Description |
|---|---|
| tests/test_presets.py | Adds targeted warning filters to keep self-test preset composition tests quiet while preserving visibility of other warnings. |
Copilot's findings
Tip
Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comments suppressed due to low confidence (1)
tests/test_presets.py:2207
- Same as above: these filterwarnings patterns omit the warning category, which makes the ignore broader than necessary. Adding
:UserWarningwould keep the suppression tightly scoped to the expected warnings.
@pytest.mark.filterwarnings(
r"ignore:Cannot compose command 'speckit\.wrap-test'"
)
@pytest.mark.filterwarnings(
r"ignore:Post-install reconciliation failed for self-test"
)
- Files reviewed: 1/1 changed files
- Comments generated: 1
Address Copilot review on github#2373: the previous filterwarnings entries omitted the warning category, so any warning class with a matching message would have been silenced. Append :UserWarning to the four filters so only the deliberately-emitted UserWarnings from _reconcile_composed_commands() are ignored.
There was a problem hiding this comment.
Copilot's findings
Tip
Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- Files reviewed: 1/1 changed files
- Comments generated: 0 new
# Conflicts: # tests/test_presets.py
|
Rebased onto latest |
There was a problem hiding this comment.
Copilot's findings
Tip
Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- Files reviewed: 1/1 changed files
- Comments generated: 1
| @pytest.mark.filterwarnings( | ||
| r"ignore:Cannot compose command 'speckit\.wrap-test':UserWarning" | ||
| ) | ||
| @pytest.mark.filterwarnings( | ||
| r"ignore:Post-install reconciliation failed for self-test:UserWarning" | ||
| ) |
|
Please address Copilot feedback. If not applicable, please explain why |
Address Copilot feedback: the class-level @pytest.mark.filterwarnings on TestPresetSkills was too broad. The 'Post-install reconciliation failed' filter could mask real reconciliation regressions, since that warning is only emitted when _reconcile_composed_commands/_reconcile_skills raises. Tests in TestPresetSkills already call install_self_test_preset(), which scopes a narrow filter to the expected wrap-strategy 'Cannot compose' warning. The class-level filters are redundant for those calls and unsafe elsewhere, so they are removed.
|
@mnriem! Addressed Copilot's feedback in 6d3d6de. The class-level |
There was a problem hiding this comment.
Copilot's findings
Tip
Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- Files reviewed: 1/1 changed files
- Comments generated: 1
| """Tests using the self-test preset that ships with the repo. | ||
|
|
||
| The self-test preset ships a wrap-strategy command (``speckit.wrap-test``) | ||
| without a corresponding core base layer; reconciliation deliberately | ||
| surfaces a UserWarning in that case. The filters above acknowledge those | ||
| expected warnings so the suite stays quiet without masking unrelated ones. | ||
| """ |
|
Please address Copilot feedback |
…iltering Address Copilot feedback: docstring referred to 'filters above', but the fix uses warnings.filterwarnings inside install_self_test_preset rather than class-level decorators. Updated the docstring to describe the actual mechanism.
Summary
Fixes #2363 —
tests/test_presets.pyproduces 18 unhandledUserWarnings when the self-test preset is installed duringtests.
Root cause
The repo's
presets/self-test/preset providesspeckit.wrap-test, a wrap-strategy command intentionally without abase layer to exercise that code path.
_reconcile_composed_commands()inpresets.py:786deliberately emitsUserWarning("Cannot compose command 'speckit.wrap-test': no base layer...")and a follow-up"Post-install reconciliation failed..."warning — these are expected for this fixture but were not acknowledged at the test level.Fix
Per the issue's option #2, add
@pytest.mark.filterwarnings(...)decorators onTestSelfTestPresetandTestPresetSkillsfor the two known messages. OtherUserWarningsources still propagate normally so futureregressions remain visible.
Verification